Learning Knowledge Graph Embeddings for Natural Language Processing

Knowledge graph embeddings provide powerful latent semantic representation for the structured knowledge in knowledge graphs, which have been introduced recently. Being different from the already widely-used word embeddings that are conceived from plain text, knowledge graph embeddings enable direct explicit relational inferences among entities via simple calculation of embedding vectors. In particular, they are quite effective at highlighting key concepts underlying sophisticated human languages. Therefore knowledge graph embeddings provide potent tools for modern NLP applications, inasmuch as they well-preserve the multi-faceted knowledge and structures of the knowledge graphs. However, recent research efforts have not progressed much beyond representing simple or multi-mapping relations (e.g. one-to-many, many-to-many) for monolingual knowledge graphs. Many crucial problems, including how to preserve important relational properties, and how to characterize both monolingual and cross-lingual knowledge in multiple language-specific versions of the knowledge bases, still remain largely unsolved. Another pressing challenge is, how to incorporate knowledge graph embeddings into NLP tasks which currently rely on word embeddings or other representation techniques. In this prospectus, we first propose new models for encoding the multi-faceted knowledge as stated. We start from investigating the approach that captures cross-lingual transitions across difference language-specific versions of embedding spaces, while in each embedding space the monolingual relations are well-preserved. We then study the approach to retain the important relational properties that commonly exist in domain-specific and ontology-level knowledge graphs, including transitivity, symmetry, and hierarchies. After that, we explore how our new embedding models may be used to improve modern NLP tasks, including relation extraction, knowledge alignment, semantic relatedness analysis, and sentiment analysis.